Systems and methods of using spatial/spectral/temporal imaging for hidden or buried explosive detection

ABSTRACT

A method and system for increasing the detection, location, identification or classification of objects hidden on the surface or buried below the surface of the ground is disclosed. The method acquires image data in separate IR and/or visible spectral regions simultaneously and converts the data into intensity value arrays for each spectral region. These intensity value arrays are transformed into two-dimensional discrete wavelet transform arrays for each spectral region. The background clutter from the two-dimensional discrete wavelet transform arrays is removed; forming clutter reduced two-dimensional discrete wavelet transform arrays. The inverse two-dimensional discrete wavelet transform is performed on the clutter reduced two-dimensional discrete wavelet transform arrays to form clutter removed intensity value arrays. These arrays are subtracted in a pair-wise mariner to obtain chemical-specific spectral signatures. The processed images are correlated with 3-dimensional matched filters of known emissive signatures of objects to detect the presence of the object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/112,245 titled “Methods of UsingSpatial/Spectral/Temporal Imaging for Hidden or Buried ExplosiveDetection,” filed Nov. 6, 2008, the contents of which are herebyincorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure is in the field of electronic imaging. Moreparticularly in the area of multispectral surveillance imaging systemsfor identifying explosives hidden on the surface of or buried under theground.

BACKGROUND OF THE INVENTION

The detection and identification of explosives hidden on the surface orburied under the ground has a multitude of commercial applications.After Sep. 11, 2000 the United States proclaimed war on the terroristsresponsible for the attack on the Twin Towers in New York City. Thisincluded those directly involved as well as those who actively aided inthe attack. Military intelligence from a number of sources identifiedspecific individuals including the then ruler of Iraq. In previousUnited Nations investigations of Iraq sanctions were instituted againstthe country and its ruler to force compliance with international nuclearweapons reduction treaties. When attempts failed the United Statesinvaded Iraq. During the attack and during the Iraq occupationnon-conventional weapons were used by the insurgence to retaliateagainst United States forces. These devices were called improvisedexplosive devices or “IED”s. Because of their non-conventionalappearance they were difficult to identify and consequently caused anumber of casualties. Currently a number of methods exist for thedetection of IED's including (1) Ground Penetrating Radar, which doesnot meet performance targets established by the military, (2) ElectricalImpedance Tomography, which requires physical contact with the groundand which cannot be used in very dry soil, (3) X-Ray Backscatter, whichis physically large, consumes large power and which requires high X-raypower to operate, (4) Acoustic-Seismic device, which is not effectivewhen surface vegetation is present, (5) Chemical Vapor detection system,which is not sensitive to very low chemical density, (6) Biologicalmethod where mammals like dogs do not perform well under fatigue, (7)Nuclear Quadrupole Resonance, which is effective only at highsignal-to-noise ratios, (8) Prodders and Probes, which are risky due tothe possibility of detonation, (9) Thermal Signature Detection usinganalytical models, which requires soil-specific parameters such as heatcapacity, soil density and thermal conductivity and which requiresintensive iterative computations that may be unstable. Most of theseexisting methods either make physical contact with the ground or are notreliable.

Consequently there is a need for a multispectral imaging system that candetect, identify, locate and classify explosives such as improvisedexplosive devices (IEDs) both hidden on the surface or buried below thesurface of the ground that is more reliable than current methods, doesnot require substantial amounts of energy to operate and does notrequire physical contact with the soil.

SUMMARY OF THE INVENTION

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the claimed subject matter. Thissummary is not an extensive overview, and is not intended to identifykey/critical elements or to delineate the scope of the claimed subjectmatter. Its purpose is to present some concepts in a simplified form asa prelude to the more detailed description that is presented later.

In one aspect of the present disclosure, a method for detecting,identifying or classifying of objects hidden on the surface or buriedbelow the surface of the ground is provided, comprising: acquiring imagedata in separate infrared (IR) and/or visible spectral regionssimultaneously; converting the acquired image data into intensity valuearrays for each spectral region; applying a transformation on theintensity value arrays to form two-dimensional discrete wavelettransform arrays for each spectral region; removing background clutterfrom the two-dimensional discrete wavelet transform arrays to formclutter reduced two-dimensional discrete wavelet transform arrays;performing an inverse two-dimensional discrete wavelet transform on theclutter reduced two-dimensional discrete wavelet transform arrays toform clutter removed intensity value arrays; subtracting the clutterremoved intensity value arrays in a pair-wise manner to obtainchemical-specific spectral signatures; and correlating thechemical-specific spectral signatures with 3-dimensional matched filtersof known emissive signatures of objects to detect a presence of theobject.

In another aspect of the present disclosure, the above method isprovided, further comprising: applying at least one of a 3- and4-dimensional feature space to compare a thermal spectral signature ofan inspected soil with a thermal spectral signature of surrounding soil,to determine a disturbed soil condition.

To the accomplishment of the foregoing and related ends, certainillustrative aspects are described herein in connection with thefollowing description and the annexed drawings. These aspects areindicative, however, of but a few of the various ways in which theprinciples of the claimed subject matter may be employed and the claimedsubject matter is intended to include all such aspects and theirequivalents.

Other advantages and novel features may become apparent from thefollowing detailed description when considered in conjunction with thedrawings. As such, other aspects of the disclosure are found throughoutthe specification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of capturing spatial/spectral/temporalsignature of explosive chemical;

FIG. 2 is an illustration of image acquisition and processing; and

FIG. 3 is an illustration of clusters of soil constituents in a3-dimensional feature space.

DETAILED DESCRIPTION OF THE INVENTION

The present disclosure provides methods for detecting explosive hiddenabove or buried under the ground, such as IEDs, exploiting the thermalemissivity of chemical explosives. Each chemical explosive has adistinct emissivity characteristic or signature over a particularspectral region. This spectral signature falls in the long wave infrared(LWIR) region. Spectral signatures of explosives also change with time.The present disclosure detects and classifies buried IEDs using thisthermal detection. An infrared detector array captures the thermalemissivity of the soil and its constituents. In order to distinguish thechemical signature from background clutter, we use infrared detectorarray as it is more suited to the spectral range of interest. Thisdisclosure is unique in that it utilizes spatial, temporal and twospectral bands—a total of four dimensions to detect as much informationabout the chemical and background signatures as possible to maximize theprobability of detection and minimize false alarm rates.

FIG. 1 depicts one embodiment of the present disclosure. The systemcaptures the spectral signatures of the land surface area being imagedin two distinct bands of the electromagnetic spectrum. The land surfaceis scanned over a certain area at a fixed rate. The cylindrical lensfocuses the scene on to the detector array of size M pixels by N rows.The focused thermal image is split by a beam splitter. Each beam thenpasses through a prism, which decomposes the corresponding thermalsignal into its spectral components and the spectral components arecaptured by the detector array. Thus, each column of the detector arrayrepresents the emissive spectral signature of the land surface area thatis being imaged on to a pixel. This process of capturing emissivesignatures is carried out in time sequence thereby acquiringspatial-spectral-temporal information of the land surface area ofinterest.

FIG. 2 is a schematic diagram showing signal acquisition and processing.Each pixel value is subtracted from the background value and thedifference signal is digitized by an analog-to-digital converter (ADC).This scheme of analog-to-digital conversion improves the accuracy ofrepresenting a pixel value in digital form with a fixed number of bitsof representation. The same procedure is used for the pixels in thesecond spectral band, as shown. After the signal is acquired, the imagesfrom the two spatial-spectral-temporal dimensions are processed eitherby a special purpose signal processing hardware or by a computer (PC ormainframe). The processing consists of optimal weighted differencingwhose purpose is to eliminate background clutter and extractchemical-specific spectral signatures. After signature extraction,detection and classification tasks are performed.

Detection is achieved by correlating the processed images with3-dimensional matched filters. These matched filters are known a priorias they correspond to the emissive signatures of known chemicalexplosives. If the form of these filters are known analytically (forexample if they are treated as Gaussian functions), then the filterparameters such as the mean and standard deviation are known or may beestimated from the acquired thermal images. The processed images arecorrelated with all the matched filters from a database andcorresponding test statistics are computed. These test statistics areused in a maximum a posteriori (MAP) sense to test the hypotheses todecide whether an IED is present or not.

Classification of the detected chemical explosives is conducted using asuitable clustering procedure. In the 3- or 4-dimensional feature space,each soil constituent forms a cluster that is distinguished from theothers, as shown in FIG. 3. In order to classify the detected chemicalexplosives, we compute the feature vectors of the emissive signaturesand use the maximum likelihood (ML) method to identify the correspondingclasses.

Our current disclosure exploits the following features of chemicalexplosives: (1) Disturbance of soil with respect to thesurrounding—digging and placing explosive chemicals in the soil changesthe appearance and is captured by both visible and infrared detectors.(2) Altered thermal signature—changes the thermal signature of soil byamounts exceeding 0.1° K. (3) Thermal emissive spectral signature—soildisturbance enhances spectral contrast. Because of these informativefeatures used for detection and classification, the present disclosureis more robust and reliable as compared to the currently used methods.Moreover, the present disclosure can be mast-mounted on a vehicle ormounted on a helicopter and has little risk of detonation, covers alarge area and can be carried out covertly. Additionally, the presentdisclosure uses (I) custom sensor in the visible spectrum to give astrong signal of the land surface, (II) focal plane arrays for the nearIR spectrum at room temperature, (III) cantilever technology for thesensor in the LWIR spectrum. It should be pointed out that the infraredsensor using cantilever technology is readily available in the marketand that it requires no cooling as opposed to other systems whichrequire cooling by liquid nitrogen.

Camera

The multispectral surveillance system (MLSS) is a system for thedetection of emissivity characteristics or signatures of explosives onor below the surface of the ground in the long wave infrared (LWIR)region through the elimination of most of the surface reflected thermalemission clutter components from multispectral images. The digitizedoutputs of the multispectral sensor are first captured and stored in thememory of a frame grabber card. Two of the three available spectralbands, X and Y, are selected for the processing that follows.

Sensors must exhibit a linear response to the input thermal emission;i.e., the output signal doubles for a doubling of the intensity of IRsignal on the sensor. Deviation from this behavior is a non-linearresponse, which is measured for each pixel. A set of curve fittingcoefficients is calculated for each pixel or class of pixels. Thesecoefficients are then used to calculate the correction to be applied toeach pixel during frame readout, and are stored in a ROM and tablelookup.

The algorithms, operating system, and system control files are stored inan operational file folder that is downloaded to the appropriate unitsupon power turn on of the system. The algorithms and system controlfunctions are then executed by the appropriate CPU, DSP, FPGA, or ASIC.

Use

The present disclosure is a processing method used for the detection ofexplosives hidden on the surface or buried in the ground using spectralimages. These processing methods work with either two or three componentimages. In either case, the processing methods can be classified asoptimal or suboptimal. Optimal processing methods use the statistics ofthe component images while the suboptimal ones use ad hoc parameters.FIG. 8 (former FIG. 1, patent app. 2) shows the classification of thevarious pre-detection processing methods described below. Theseprocessing methods could be performed purely in the spatial domain,frequency domain or the time and spatial domain.

In optimal processing, the camera sensors produce intensity images withdiffering intensity values in the three IR/visible spectral regionsdepending on the emissivity characteristics of the explosive, thepresence of other infrared emitting sources such as fires or geothermalareas and reflections from the ground in the area being screened. Themain idea behind explosive detection, therefore, is to subtract onecomponent image from another so that the difference image retains onlythe explosive's signature. Preferably the signal from the surfacereflection has been completely removed. It should be pointed out thatthe contrast due to the target between the component images is verysmall and depends on the emissivity of the explosives in the spectralregions. Thus, the process methods discriminate a very small contrastfrom image components. The ability to detect such a miniscule contrastdepends on 1) the sensor dynamic range, 2) effectiveness of thepreprocessing methods in eliminating background clutter, and 3)detection algorithm itself. Our processing methods address the thirdelement primarily because the first is a limitation of the sensor andthe second can be eliminated with standard methods known to thoseskilled in the art.

The processing methods of the present disclosure may be applied totwo-channel and three-channel imaging. The processing methods fortwo-channel imaging may be used to minimize the mean square error, errorvariance, correlation and/or covariance. To minimize the mean squareerror a scaled image component is subtracted from another to enhance thelow contrast of the explosive's signature. Consider the channelsspanning three contiguous IR spectral bands. For the sake of argument,let us denote the three such contiguous channel pixels at location (m,n)by R(m,n), G(m,n) and B(m,n) and name them as red, green and bluerespectively. Then, define the difference pixel e_(R)(m,n) at the samepixel location as Equation 1e _(R)(m,n)=R(m,n)−α_(R) G(m,n)We want to determine the scale factor α_(R), so as to minimize the meansquare error

$ɛ_{R} = {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{e_{R}^{2}\left( {m,n} \right)}.}}}}$The optimal coefficient α*_(R) is obtained by setting the derivative ofε_(R) with respect to α_(R), to zero. This results in the optimalcoefficient value as given by Equation 2

$\alpha_{R}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}}$Note that both the numerator and denominator have the same normalizingfactor 1/MN and can, therefore, be omitted. The numerator of Equation 2is a measure of the correlation between the two spectral bands R(m,n)and G(m,n) under consideration.

In a similar manner we can obtain the optimal scaling coefficients forthe green-blue and blue-red channels and are given, respectively, by:

$\begin{matrix}{{\alpha_{G\;}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}{\overset{M - 1}{\sum\limits_{m = 0}}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}},{\alpha_{B}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{R^{2}\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 3}\end{matrix}$

Minimization of the error variance ε_(R) instead of minimizing the meansquare error, is given by:

$\begin{matrix}\begin{matrix}{ɛ_{R} = {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left( {{e_{R}\left( {m,n} \right)} - \mu_{{cR}\;}} \right)^{2}}}}} \\{= {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left( {\left( {{R\left( {m,n} \right)} - \mu_{R\;}} \right) - {\alpha_{R}\left( {{G\left( {m,n} \right)} - \mu_{G}} \right)}} \right)^{2}}}}}\end{matrix} & {{Equation}\mspace{14mu} 4}\end{matrix}$where μ_(R) is given by:

$\begin{matrix}{\mu_{eR} = {{\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R\left( {m,n} \right)}}}} - {\frac{\alpha_{R}}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G\left( {m,n} \right)}}}}}} & {{Equation}\mspace{14mu} 5}\end{matrix}$This results in the optimal coefficient, which is expressed as:

$\begin{matrix}{\alpha_{R}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{\left( {{R\left( {m,n} \right)} - \mu_{R}} \right)\left( {{G\left( {m,n} \right)} - \mu_{G}} \right)}}}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left( {{G\left( {m,n} \right)} - \mu_{G}} \right)^{2}}}} & {{Equation}\mspace{14mu} 6}\end{matrix}$

The numerator of

Equation 6 is a measure of the covariance rather than correlation andthe denominator is the variance rather than sum of squares. Followingthe same argument for the red-green channel, we obtain the optimalcoefficients for the green-blue and blue-red channels as:

$\begin{matrix}{{\alpha_{G}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{\left( {{G\left( {m,n} \right)} - \mu_{G}} \right)\left( {{B\left( {m,n} \right)} - \mu_{B}} \right)}}}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left( {{B\left( {m,n} \right)} - \mu_{B}} \right)^{2}}}}{\alpha_{B}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{\left( {{B\left( {m,n} \right)} - \mu_{B}} \right)\left( {{R\left( {m,n} \right)} - \mu_{R}} \right)}}}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left( {{R\left( {m,n} \right)} - \mu_{R}} \right)^{2}}}}} & {{Equation}\mspace{14mu} 7}\end{matrix}$

While the above procedures determine the optimal coefficients byminimizing the mean square error or the error variance, there exists analternative minimization criterion to compute the optimal weights. Thisis the minimum correlation between the difference channels. Consider thetwo error images corresponding to the red and green channels, as givenby:e _(R)(m,n)=R(m,n)−α_(R1) G(m,n), e _(G)(m,n)=G(m,n)−α_(G1)B(m,n)  Equation 8Determination of the two coefficients, α_(R1), and α_(G1), by minimizingthe cross correlation is expressed as:

$\begin{matrix}\begin{matrix}{ɛ_{RG} = {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{e_{R}\left( {m,n} \right)}{e_{G}\left( {m,n} \right)}}}}}} \\{= {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left( {{R\left( {m,n} \right)} - {\alpha_{R\; 1}{G\left( {m,n} \right)}}} \right)}}}} \\{\left( {{G\left( {m,n} \right)} - {\alpha_{G\; 1}B\left( {m,n} \right)}} \right)}\end{matrix} & {{Equation}\mspace{14mu} 9}\end{matrix}$Using the standard procedure for the minimization, the following normalequations are obtained:

$\begin{matrix}{{{\alpha_{G\; 1}^{*}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} - {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}}} = 0} & {{Equation}\mspace{14mu} 10} \\{{{\alpha_{R\; 1}^{*}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} - {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} = 0} & {{Equation}\mspace{14mu} 11} \\{{{\alpha_{B\; 1}^{*}{\overset{M - 1}{\sum\limits_{m = 0}}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} - {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}} = 0} & {{Equation}\mspace{14mu} 12} \\{{{\alpha_{G\; 1}^{*}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} - {\overset{M - 1}{\sum\limits_{m = 0}}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} = 0} & {{Equation}\mspace{14mu} 13} \\{{{\alpha_{R\; 1}^{*}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} - {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}}} = 0} & {{Equation}\mspace{14mu} 14} \\{{{\alpha_{B\; 1}^{*}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} - {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} = 0} & {{Equation}\mspace{14mu} 15}\end{matrix}$Since

-   Equation 11 and-   Equation 14 are distinct, as are the other equations, by adding them    α*_(R1), can be resolved by:

$\begin{matrix}{\alpha_{R\; 1}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}} & {{Equation}\mspace{14mu} 16}\end{matrix}$In a like manner, the optimal coefficients for the green and bluechannels are given by:

$\begin{matrix}{{\alpha_{G\; 1}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}},} & {{Equation}\mspace{14mu} 17} \\{{\alpha_{B\; 1}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}},} & {{Equation}\mspace{14mu} 18}\end{matrix}$

These are listed in Table 1.

By minimizing the covariance instead of the correlation between thechannels, we obtain another set of optimal coefficients. These areidentical in form to the coefficients in

Equation 11 and

Equation 14 except that each variable is replaced by its mean-removedvalue. These are listed in Table 1.

The processing methods for three-channel imaging may also be used tominimize the mean square error, error variance, correlation and/orcovariance. In the three-channel imaging, all the three channels areconsidered to generate the difference image. Consequently, Equation 1 isrewritten as:e _(R)(m,n)=R(m,n)−α_(R) G(m,n)−β_(R) B(m,n)  Equation 19Similarly, the other two difference channels are given by:e _(G)(m,n)=G(m,n)−α_(G) B(m,n)−β_(G) R(m,n)  Equation 20e _(B)(m,n)=B(m,n)−α_(B) R(m,n)−β_(B) G(m,n)  Equation 21Similar to the two-channel case, the minimization of the mean squareerror gives us the normal equations for the optimal coefficients:

$\begin{matrix}{\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}\end{bmatrix}{\quad{\begin{bmatrix}\alpha_{R}^{*} \\\alpha_{R}^{*}\end{bmatrix} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}\end{bmatrix}}}} & {{Equation}\mspace{14mu} 22} \\{\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}}\end{bmatrix}{\quad{\begin{bmatrix}\alpha_{G}^{*} \\\beta_{G}^{*}\end{bmatrix} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}\end{bmatrix}}}} & {{Equation}\mspace{14mu} 23} \\{{\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{B}^{*} \\\beta_{B}^{*}\end{bmatrix}} = {\quad\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}\end{bmatrix}}} & {{Equation}\mspace{14mu} 24}\end{matrix}$

The optimal coefficients for the minimum error variance case will beidentical to those in

Equation 22 through

Equation 24, for the three-channel case, except that each term must bereplaced by its mean removed value. These are listed in Table 2.

The difference channels are as defined in

Equation 19 to

Equation 21. Minimization of the pair-wise correlations is as follows.

$\begin{matrix}{{ɛ_{R} = {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{e_{R}\left( {m,n} \right)}{e_{G\;}\left( {m,n} \right)}}}}}}{ɛ_{G} = {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{{e_{G}\left( {m,n} \right)}{e_{B}\left( {m,n} \right)}}}}}}{ɛ_{B} = {\frac{1}{MN}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{e_{B}\left( {m,n} \right)}{e_{R}\left( {m,n} \right)}}}}}}} & {{Equation}\mspace{14mu} 25}\end{matrix}$By setting the derivatives of the three terms inEquation 25 with respect to the three coefficients the followingequations are obtained:

$\begin{matrix}{{{\alpha_{G}{\overset{M - 1}{\sum\limits_{m = 0}}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} + {\beta_{G}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}}} & {{Equation}\mspace{14mu} 26} \\{{{\alpha_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} + {\beta_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 27} \\{{{\alpha_{G}{\overset{M - 1}{\sum\limits_{m = 0}}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}} + {\beta_{G}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 28} \\{{{\alpha_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} + {\beta_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}}} & {{Equation}\mspace{14mu} 29} \\{{{\alpha_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} + {\beta_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}} & {{Equation}\mspace{14mu} 30} \\{{{\alpha_{G}{\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} + {\beta_{G}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 31} \\{{{\alpha_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}}} + {\beta_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 32} \\{{{\alpha_{G}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} + {\beta_{G}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}}} & {{Equation}\mspace{14mu} 33} \\{{{\alpha_{R}{\overset{M - 1}{\sum\limits_{m = 0}}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} + {\beta_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{R^{2}\left( {m,n} \right)}}}} & {{Equation}\mspace{14mu} 34} \\{{{\alpha_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} + {\beta_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{G^{2}\left( {m,n} \right)}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 35} \\{{{\alpha_{R}{\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{G^{2}\left( {m,n} \right)}}}} + {\beta_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}} & {{Equation}\mspace{14mu} 36} \\{{{\alpha_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}} + {\beta_{B}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{B^{2}\left( {m,n} \right)}}}} & {{Equation}\mspace{14mu} 37}\end{matrix}$Since equations (29) and (34) are identical we retain one of them,Equations (27) and (36) involve the same set of variables but aredistinct. Therefore, we form a new equation by subtracting (27) from(36) to get:

$\begin{matrix}{{{\alpha_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{G^{2}\left( {m,n} \right)} - {{G\left( {m,n} \right)}{B\left( {m,n} \right)}}} \right\rbrack}}} + {\beta_{R}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{{G\left( {m,n} \right)}{B\left( {m,n} \right)}} - {B^{2}\left( {m,n} \right)}} \right\rbrack}}}} = {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{{R\left( {m,n} \right)}{G\left( {m,n} \right)}} - {{B\left( {m,n} \right)}{R\left( {m,n} \right)}}} \right\rbrack}}} & {{Equation}\mspace{14mu} 38}\end{matrix}$Now equations (29) and (38) can be consolidated as:

$\begin{matrix}{\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{RG}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{BR}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {G^{2} - {GB}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}\left\lbrack {{GB} - B^{2}} \right\rbrack}}\end{bmatrix}{\quad{\begin{bmatrix}\alpha_{R} \\\beta_{R}\end{bmatrix} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}R^{2}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{RG} - {BR}} \right\rbrack}}\end{bmatrix}}}} & {{Equation}\mspace{14mu} 39}\end{matrix}$In Equation (39), the double subscripts are omitted for want of space.In a similar manner, we arrive at the rest of the equations:

$\begin{matrix}{{\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{GB}}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{RG}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {B^{2} - {BR}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{BR} - R^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{G\;} \\\beta_{G}\end{bmatrix}} = {\quad\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}G^{2}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{GB} - {RG}} \right\rbrack}}\end{bmatrix}}} & {{Equation}\mspace{14mu} 40} \\{\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}{BR}}} & {\sum\limits_{m = 0}^{M - 1}{\overset{N - 1}{\sum\limits_{n = 0}}{GB}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {R^{2} - {RG}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{RG} - G^{2}} \right\rbrack}}\end{bmatrix}{\quad{\begin{bmatrix}\alpha_{B} \\\beta_{B}\end{bmatrix} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}B^{2}}} \\{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\left\lbrack {{BR} - {GB}} \right\rbrack}}\end{bmatrix}}}} & {{Equation}\mspace{14mu} 41}\end{matrix}$The optimal coefficients for the three-channel minimum correlation caseare obtained by solving the normal equations (39) through (41).

For the minimum covariance case the optimal coefficients will beidentical to those corresponding to the minimum correlation case, exceptthat each entry in the

Equation 39 through

Equation 41 must be replaced by its mean-removed value. These are listedin Table 2.

TABLE 1 Optimal Coefficients: Two-channel case Case Optimal CoefficientsMinimum MSE or Variance Value Minimum MSE$\alpha_{R}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{G^{2}\left( {m,n} \right)}}}$${\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{R^{2}\left( {m,n} \right)}}} - \frac{\left\{ {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} \right\}^{2}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{G^{2}\left( {m,n} \right)}}}$$\alpha_{G}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{B^{2}\left( {m,n} \right)}}}$${\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{G^{2}\left( {m,n} \right)}}} - \frac{\left\{ {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} \right\}^{2}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{B^{2}\left( {m,n} \right)}}}$$\alpha_{B}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{R^{2}\left( {m,n} \right)}}}$${\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{B^{2}\left( {m,n} \right)}}} - \frac{\left\{ {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} \right\}^{2}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{R^{2}\left( {m,n} \right)}}}$Minimum Variance$\alpha_{R}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}}^{2}\left( {m,n} \right)}}}$$\sigma_{R}^{2} - \frac{\left\{ {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}} \right\}^{2}}{\sigma_{G}^{2}}$$\alpha_{G}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}}^{2}\left( {m,n} \right)}}}$$\sigma_{G}^{2} - \frac{\left\{ {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}} \right\}^{2}}{\sigma_{B}^{2}}$$\alpha_{G}^{*} = \frac{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}}^{2}\left( {m,n} \right)}}}$$\sigma_{B}^{2} - \frac{\left\{ {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}} \right\}^{2}}{\sigma_{R}^{2}}${tilde over (R)}(m, n) = R(m, n) − μ_(R), {tilde over (G)}(m, n) = G(m,n) − μ_(G) {tilde over (B)}(m, n) = B(m, n) − μ_(B) Case OptimalCoefficients: 1^(st) set Remark Minimum Correlation$\alpha_{R}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{R^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}$$\alpha_{G}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{G^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}}$$\alpha_{B}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{B^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}}$Minimum Covariance$\alpha_{R}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}}^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}}}${tilde over (R)}(m, n) = R(m, n) − μ_(R)$\alpha_{G}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}}^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}}}${tilde over (G)}(m, n) = G(m, n) − μ_(G)$\alpha_{B}^{*} = \frac{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}}^{2}\left( {m,n} \right)}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}}}{{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}} + {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}}}${tilde over (B)}(m, n) = B(m, n) − μ_(B)

TABLE 2 Optimal Coefficients: Three-channel case Case OptimalCoefficients Minimum MSE ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{G^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{B^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{R} \\\beta_{R}\end{bmatrix}} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{B^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{R^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{G} \\\beta_{G}\end{bmatrix}} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{R^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{R\left( {m,n} \right)}{G\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{G^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{B} \\\beta_{B}\end{bmatrix}} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{B\left( {m,n} \right)}{R\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{G\left( {m,n} \right)}{B\left( {m,n} \right)}}}}\end{bmatrix}$ Minimum Variance ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}}^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}}^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{R} \\\beta_{R}\end{bmatrix}} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}}^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}}^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{G} \\\beta_{G}\end{bmatrix}} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}}^{2}\left( {m,n} \right)}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{R}\left( {m,n} \right)}{\overset{\sim}{G}\left( {m,n} \right)}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}}^{2}\left( {m,n} \right)}}}\end{bmatrix}\begin{bmatrix}\alpha_{R} \\\beta_{R}\end{bmatrix}} = \begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{B}\left( {m,n} \right)}{\overset{\sim}{R}\left( {m,n} \right)}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{{\overset{\sim}{G}\left( {m,n} \right)}{\overset{\sim}{B}\left( {m,n} \right)}}}}\end{bmatrix}$ {tilde over (R)}(m, n) = R(m, n) − μ_(R), {tilde over(G)}(m, n) = G(m, n) − μ_(G), {tilde over (B)}(m, n) = B(m, n) − μ_(B)Minimum Correlation ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{RG}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{BR}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {G^{2} - {GB}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{GB} - B^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{B} \\\beta_{B}\end{bmatrix}} = \begin{bmatrix}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\; R^{2}}}\;} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{RG} - {BR}} \right\rbrack}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{GB}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{RG}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {B^{2} - {BR}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{BR} - R^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{G} \\\beta_{G}\end{bmatrix}} = \begin{bmatrix}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\; G^{2}}}\;} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{GB} - {RG}} \right\rbrack}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{BR}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{GB}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {R^{2} - {RG}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{RG} - G^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{B} \\\beta_{B}\end{bmatrix}} = \begin{bmatrix}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\; B^{2}}}\;} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{BR} - {GB}} \right\rbrack}}\end{bmatrix}$ Minimum Covariance ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{R}\overset{\sim}{G}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{B}\overset{\sim}{R}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{G}}^{2} - {\overset{\sim}{G}\overset{\sim}{B}}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{G}\overset{\sim}{B}} - {\overset{\sim}{B}}^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{R} \\\beta_{R}\end{bmatrix}} = \begin{bmatrix}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{R}}^{2}}}\;} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{R}\overset{\sim}{G}} - {\overset{\sim}{B}\overset{\sim}{R}}} \right\rbrack}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{G}\overset{\sim}{B}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{R}\overset{\sim}{G}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{B}}^{2} - {\overset{\sim}{B}\overset{\sim}{R}}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{B}\overset{\sim}{R}} - {\overset{\sim}{R}}^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{G} \\\beta_{G}\end{bmatrix}} = \begin{bmatrix}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{G}}^{2}}}\;} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{G}\overset{\sim}{B}} - {\overset{\sim}{R}\overset{\sim}{G}}} \right\rbrack}}\end{bmatrix}$ ${\begin{bmatrix}{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{B}\overset{\sim}{R}}}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{G}\overset{\sim}{B}}}} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{R}}^{2} - {\overset{\sim}{R}\overset{\sim}{G}}} \right\rbrack}} & {\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{R}\overset{\sim}{G}} - {\overset{\sim}{G}}^{2}} \right\rbrack}}\end{bmatrix}\begin{bmatrix}\alpha_{B} \\\beta_{B}\end{bmatrix}} = \begin{bmatrix}{{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n = 0}^{N - 1}\;{\overset{\sim}{B}}^{2}}}\;} \\{\sum\limits_{m = 0}^{M - 1}\;{\sum\limits_{n = 0}^{N - 1}\;\left\lbrack {{\overset{\sim}{B}\overset{\sim}{R}} - {\overset{\sim}{G}\overset{\sim}{B}}} \right\rbrack}}\end{bmatrix}$ {tilde over (R)}(m, n) = R(m, n) − μ_(R), {tilde over(G)}(m, n) = G(m, n) − μ_(G), {tilde over (B)}(m, n) = B(m, n) − μ_(B)

In suboptimal processing there is no cost function involved, thecoefficient value is chosen on an ad hoc basis and the resulting meansquare error or error variance is not necessarily the minimum. Further,only two components are used in the differencing process.

The difference images are obtained by selecting the weights as the ratioof the mean values of the respective image components involved, as givenby:

$\begin{matrix}{{{{e_{R}\left( {m,n} \right)} = {{R\left( {m,n} \right)} - {\alpha_{R\;}{G\left( {m,n} \right)}}}};}{\alpha_{R} = \frac{\mu_{R}}{\mu_{G}}}} & {{Equation}\mspace{14mu} 42} \\{{{{e_{G}\left( {m,n} \right)} = {{G\left( {m,n} \right)} - {\alpha_{G}{B\left( {m,n} \right)}}}};}{\alpha_{G} = \frac{\mu_{G}}{\mu_{B}}}} & {{Equation}\mspace{14mu} 43} \\{{{{e_{B}\left( {m,n} \right)} = {{B\left( {m,n} \right)} - {\alpha_{B}{R\left( {m,n} \right)}}}};}{\alpha_{B} = \frac{\mu_{B}}{\mu_{R\;}}}} & {{Equation}\mspace{14mu} 44}\end{matrix}$In the second case, the coefficients are chosen as the ratio of thestandard deviations of the component images and the weights are givenby:

$\begin{matrix}{{\alpha_{R} = \frac{\sigma_{R}}{\sigma_{G\;}}};{\alpha_{G} = \frac{\sigma_{G}}{\sigma_{B}}};{\alpha_{B} = \frac{\sigma_{B}}{\sigma_{R}}}} & {{Equation}\mspace{14mu} 45}\end{matrix}$For a third case we choose the ratio of the maximum component values asthe coefficient values. Thus,

$\begin{matrix}{{{\alpha_{R} = \frac{\max\left( {R\left( {m,n} \right)} \right)}{\max\left( {G\left( {m,n} \right)} \right)}};{\alpha_{G} = \frac{\max\left( {G\left( {m,n} \right)} \right)}{\max\left( {B\left( {m,n} \right)} \right)}};}{\alpha_{B} = \frac{\max\left( {B\left( {m,n} \right)} \right)}{\max\left( {R\left( {m,n} \right)} \right)}}} & {{Equation}\mspace{14mu} 46}\end{matrix}$

The image may be processed in whole or in sub-blocks. The pre-detectionprocessing methods just described can be applied to the whole image orthey can be applied to sub-blocks of an image. Images in general andvideo sequence images in particular are non-stationary. This impliesthat the statistics change over regions within an image. To exploitnon-stationarity of images, it is advantageous to apply thepre-detection processing methods to subimages. As shown in FIG. 1, onecan process sub-blocks of fixed size or variable size or one row at atime. In processing sub-blocks of varying size, one has to adopt asuitable criterion for choosing sub-block size. This is described in thenext subsection.

Quad-tree decomposition is an efficient procedure to decompose an imageinto sub-blocks. The only requirement is that the image height and widthbe integer powers of 2. In this procedure one starts with a given imageas the root node of a tree and a suitable metric for the whole image iscalculated. Next the metrics for the four sub-blocks, each of size¼^(th) the size of the whole image are computed. A decision is made tosplit each sub-block sequentially depending on whether that sub-blockmetric is greater than the metric of its parent node. This process iscontinued until the specified sub-block size is reached. The minimumnumber of sub-blocks is 4. FIG. 2 shows a typical quad-tree structure.

The contrast of the processed component images can be improved bycontrast stretching. The contrast regions are automatically selectedusing the image statistics such as the image histogram.

Once the images are processed we need to draw the boundaries of thetargets present in the scene for further data collection such as targetclassification. This involves first smoothing of the processed images.Smoothing generally tends to blur the image. Hence we use special,non-blurring filtering such as the diffusion filtering to smooth theprocessed images. Next an edge detection processing method, typicallyCanny's method, is used followed by morphological processing tocalculate properties of the object boundaries.

The acquired raw images usually contain a number of pixels that aresaturated. These may occur as isolated single pixels or as clusters ofpixels scattered over the image. These clusters of saturated pixels areknown as glints. The presence of glints reduces the contrast of areas ofinterest, especially if the glints occupy a sizeable fraction of theimage area. Moreover, glints dominate the statistical quantities ofinterest due to their very large values. Therefore, images must bepreprocessed to remove the presence of glints. Preprocessing is alsonecessary to reduce image noise.

Preprocessing images in the wavelet domain is effective in removingglints and noise from the images. It also has the advantage ofcomputational savings. Specifically, applications of two-dimensionalDiscrete Wavelet Transform (DWT) once to the image decomposes it intoone approximation and three detail coefficients, each ¼^(th) the size ofthe original image. It has been found that biorthogonal wavelet performsbetter than orthogonal wavelet. The approximation coefficients containthe information in the original image at a lower resolution. Hence glintremoval operation is performed on the approximation coefficients. Aspointed out, since the approximation component of the DWT coefficientsis only ¼^(th) the size of the original image, computational saving isachieved. As the detail coefficients carry edge information in theoriginal image at lower resolutions, image denoising (a term used toimply the removal of noise) is effected in these components. Once glintremoval and denoising operations are performed, the image isreconstructed by applying inverse 2D DWT (IDWT) to the DWT coefficients.

The detail components of the 2D DWT of each channel image contain edgeinformation as well as noise. Noise coefficients have amplitudesaccording to a Laplacian distribution and are usually large while edgeshave smaller amplitudes. Thus noise is reduced by thresholding thedetail coefficients in the wavelet domain. These thresholds aredetermined based on the statistics of the detail coefficients.

Glints are specular reflections of the IR source by the surface beingimaged. Therefore, glints produce highly saturated pixels in all thechannels whose size depend on various properties of the reflectingsurface and it is random. Consequently, the approximation coefficientsof the 2D DWT must be inspected. In one approach, glints may be removedor reduced using morphological processing to determine the boundaries ofthe glint regions, which in turn are determined by hard or softthresholding. The thresholds are determined from the statistics of theapproximation coefficients of the 2D DWT. Thresholding can be performedon each channel or on the dominant channel. Thresholding yields a binaryimage from which the boundaries of the glints are calculated. Glintregions are eliminated by replacing them with corresponding mean valuesin all the channels.

Moreover, aspects of the claimed subject matter may be implemented as amethod, apparatus, or article of manufacture using standard programmingand/or engineering techniques to produce software, firmware, hardware,or any combination thereof to control a computer or computing componentsto implement various aspects of the claimed subject matter. The term“article of manufacture” as used herein is intended to encompass acomputer program accessible from any computer-readable device, carrier,or media. For example, computer readable media can include but are notlimited to magnetic storage devices (e.g., hard disk, floppy disk,magnetic strips . . . ), optical disks (e.g., compact disk (CD), digitalversatile disk (DVD) . . . ), smart cards, and flash memory devices(e.g., card, stick, key drive Of course, those skilled in the art willrecognize many modifications may be made to this configuration withoutdeparting from the scope or spirit of what is described herein.

What has been described above includes examples of one or moreembodiments. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the aforementioned embodiments, but one of ordinary skill inthe art may recognize that many further combinations and permutations ofvarious embodiments are possible. Accordingly, the described embodimentsare intended to embrace all such alterations, modifications andvariations that fall within the spirit and scope of the appended claims.Furthermore, to the extent that the term “includes” is used in eitherthe detailed description or the claims, such term is intended to beinclusive in a manner similar to the term “comprising” as “comprising”is interpreted when employed as a transitional word in a claim.

1. A method for detecting, identifying or classifying of objects hiddenon the surface or buried below the surface of the ground, comprising:acquiring image data in separate infrared (IR) and/or visible spectralregions simultaneously; converting the acquired image data intointensity value arrays for each spectral region; applying atransformation on the intensity value arrays to form two-dimensionaldiscrete wavelet transform arrays for each spectral region; removingbackground clutter from the two-dimensional discrete wavelet transformarrays to form clutter reduced two-dimensional discrete wavelettransform arrays; performing an inverse two-dimensional discrete wavelettransform on the clutter reduced two-dimensional discrete wavelettransform arrays to form clutter removed intensity value arrays;subtracting the clutter removed intensity value arrays in a pair-wisemanner to obtain chemical-specific thermal spectral signatures; andcorrelating the chemical-specific thermal spectral signatures with3-dimensional matched filters of known emissive signatures of objects todetect a presence of the object.
 2. The method of claim 1, furthercomprising: applying at least one of a 3-dimensional, a 4-dimensional ora 3- and 4-dimensional feature vector space to compare a thermalspectral signature of an inspected soil with a thermal spectralsignature of surrounding soil, to determine a disturbed soil condition.